Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Kestrel connection metrics with error reasons #55565

Merged
merged 43 commits into from
Jul 22, 2024

Conversation

JamesNK
Copy link
Member

@JamesNK JamesNK commented May 7, 2024

Fixes #53358

This PR improves the kestrel.connection.duration metric by adding information about why the connection ended.

Tags added or changed:

  • http.connection.protocol_code - Standards based, publicly communicated code for the connection ending. Often sent to the client. The value is either the HTTP/2 error code or HTTP/3 error code that the connection ended with. Doesn't apply to HTTP/1.1.
  • kestrel.connection.end_reason - A more detailed internal reason about why a connection ended. Always present.
  • error.type - If the end reason is an error, e.g. connection killed because of timeout, excessive stream resets, transport ended while there are active HTTP requests, invalid frame, etc. error.type could be the same as the end reason. The value could also be an exception type name if an unhandled exception is thrown from connection middleware pipeline. The first error recorded wins. Only present if there is an error.

We need to decide what the new tags is. Is it the end reason or error reason? For example, a connection can be ended by the client sending a go away frame. Do we want to track that? It's not an error to close a connection like that, so in this case we're are tracking all end reasons. On the other hand, we might want to only track unexpected and unhealthy reasons that end the connection. The benefit of only tracking errors is it is very easy to get just the unhealthy connections by filtering metrics to ones that have this tag. However, I think we can achieve this by also putting error reasons into the error.type tag.

tldr kestrel.connection.end_reason is always present and provides the reason, error.type is present if the end reason is considered an error and has the same value as kestrel.connection.end_reason.

Questions for folks:

  • Is having information about all connection end reasons useful? If it isn't then we could remove kestrel.connection.end_reason and just have error.type.
  • What end reasons are an error? For example, are the following errors or not...
    • Transport closing when there are no active HTTP requests
    • Transport closing when there are active HTTP requests
    • Connection closing because the application is shutting down
    • Client sends GOAWAY with error code of NO_ERROR
    • Client sends GOAWAY with error code of INTERNAL_ERROR

Tasks:

  • Add to HTTP/1.1
  • Add to HTTP/2
  • Add to HTTP/3
  • Add to HTTPS middleware
  • Tests
  • Bike shed tag and enum names
  • Review reasons are correct
  • Validate metrics output useful data in some known situations

cc @noahfalk @tarekgh @lmolkova @samsp-msft

@JamesNK JamesNK added the area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions label May 7, 2024
@JamesNK JamesNK requested a review from amcasey May 7, 2024 03:31
@@ -1727,7 +1727,7 @@ public async Task AbortConnectionAfterTooManyEnhanceYourCalms()
await WaitForConnectionErrorAsync<Http2ConnectionErrorException>(
ignoreNonGoAwayFrames: true,
expectedLastStreamId: int.MaxValue,
expectedErrorCode: Http2ErrorCode.INTERNAL_ERROR,
expectedErrorCode: Http2ErrorCode.ENHANCE_YOUR_CALM,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amcasey I added a HttpConnection.Abort overload that has an Http2ErrorCode argument. When there are excessive resets I made the passed in value ENHANCE_YOUR_CALM instead of INTERNAL_ERROR. That seems a more appropriate error.

Did you choose INTERNAL_ERROR for a reason here, or is it returned because that is what abort already sent?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a hotfix, so I suspect I was making the smallest change possible, but I no longer recall.

@JamesNK JamesNK force-pushed the jamesnk/connection-protocol-code branch from 7c05f97 to 8a35061 Compare May 9, 2024 08:36
@noahfalk
Copy link
Member

noahfalk commented May 9, 2024

Is having information about all connection end reasons useful?

What was the motivation to add the data initially? I can hypothesize a reason someone might find it useful (diagnosing why connections didn't live as long as they were expected to live) but I don't know how often someone needs to diagnose that issue in practice and what alternatives would they have available if this metric attribute didn't exist?

What end reasons are an error? For example, are the following errors or not..

Any attempt at classification gets very subjective but a few suggestions at least:

  1. Ideally errors represent situations that do not occur frequently and are undesirable. For example "Connection closing because the application is shutting down" might be infrequent but it doesn't sound undesirable.
  2. Ideally metric error classifications would consistent with other observable behavior that implies an error classification. For example if in-proc code throws an Exception or some error code is transmitted in the wire protocol then it would consistent for the metrics to also classify that condition as an error. If "Client sends GOAWAY with error code of NO_ERROR" then it would consistent for metrics also not to treat that as an error.

@lmolkova
Copy link

lmolkova commented May 9, 2024

I think it's valuable to know the end reason even if it's not an error and even if we don't have a specific scenario in mind.
I also don't see any drawbacks from including it - it would not increase cardinality much.

The change looks great from semconv perspective! LMK if you need any help adding new attributes to otel.

@samsp-msft
Copy link
Member

This seems like data that should be going into logs not metrics. Metric dimensions should be kept small as adding additional properties and values adds incremental storage. If we can reduce this data down to one property - was the connection closed cleanly or due to an error, and provide logs to give more details as to why. Metrics are supposed to be about aggregated statistics not detailed analysis as to what happened.

Anyone who is going to be doing analysis, can the use the metric to determine if the trend is changing, and then go to a log query to understand why.

Once exemplars are better supported in .NET/OTel they will provide the correlation between metrics and traces/logs.

@JamesNK
Copy link
Member Author

JamesNK commented May 10, 2024

@samsp-msft

There is prior art for showing more error info in error.type. That tag is already on HTTP requests for server and client in spans and metrics. Reducing error info down to a boolean is optimizing too much and goes against conventions.

http.connection.protocol_code (exact name TBD) is the HTTP connection version of request's HTTP status code. I don't think it is controversial.

kestrel.connection.end_reason is more debatable. My feeling from talking to people using Kestrel over the years is connections are a black box and people want more insight here. If there are performance/technical reasons why an extra tag is problematic, then I can reach out to ops folks to get feedback on whether end reasons for "non-error" situations would be useful.

Note that while the ConnectionEndReason enum has a lot of potential values (about 35), many of them are for protocol violations which will almost never happen, or are specific to one particular HTTP protocol (e.g. closing a critical stream is specific to HTTP/3). There are about half a dozen would occur in typical usage:

  • ConnectionReset
  • UnexpectedError
  • AbortedByApplication
  • ApplicationShutdown
  • ServerTimeout
  • IOError
  • ClientGoAway
  • TransportCompleted

That is similar to how there are many HTTP status codes, but most of them you'll never see.

src/Servers/Kestrel/Core/src/Internal/HttpConnection.cs Outdated Show resolved Hide resolved
@@ -72,7 +72,7 @@ protected override Task OnConsumeAsync()
_context.ReportApplicationError(connectionAbortedException);

// Have to abort the connection because we can't finish draining the request
_context.StopProcessingNextRequest();
_context.StopProcessingNextRequest(ConnectionEndReason.AbortedByApplication);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels like we might want to distinguish between "the application requested an abort" and "the application failed and we had to abort".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with AbortedByApp because the log message says the app aborted the connection. I agree that the reason isn't clear.

What do you think about the logging and end reason here @halter73 @Tratcher @mgravell ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the assumption here is that the only reason TryReadInternal would throw an InvalidOperationException is because middleware exited after calling HttpContext.Request.BodyReader.ReadAsync without a corresponding call to AdvanceTo, and rather than try to recover we bail out since we assume it's an unlikely scenario anyway.

I should have created a custom message explaining this rather than reuse CoreStrings.ConnectionAbortedByApplication for the exception passed to ReportApplicationError.

Ideally, we'd use "application_abort" only for HttpContext.Abort() on HTTP/1.1 connections, and then use "application_left_bodyreader_in_invalid_state" for this, and "application_canceled_response_flush" for the usage of AbortedByApplication in TimingPipeFlusher.

If we reused the same ConnectionEndReason for HttpCotnext.Abort() and this issue with leaving the body reader in an invalid state, I won't be too upset since this scenario is pretty niche. Cancelling a response body flush on an HTTP/1.1 connection probably deserves its own ConnectionEndReason though.

@@ -530,8 +544,14 @@ private void UpdateStreamTimeouts(long timestamp)
await outboundControlStreamTask;
}

// Use graceful close reason if it has been set.
if (reason == ConnectionEndReason.Unknown && _gracefulCloseReason != ConnectionEndReason.Unknown)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I don't think && _gracefulCloseReason != ConnectionEndReason.Unknown actually accomplishes anything.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean? Is it that _gracefulCloseReason != ConnectionEndReason.Unknown condition could be removed and an unknown value is set to unknown?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's been a while, but I probably meant that reason == ConnectionEndReason.Unknown implies _gracefulCloseReason != ConnectionEndReason.Unknown at this point in the code. I may well have been mistaken.

@amcasey
Copy link
Member

amcasey commented May 13, 2024

Is having information about all connection end reasons useful? If it isn't then we could remove kestrel.connection.end_reason and just have error.type.

With the notable disclaimer that I haven't actually monitored the health of a real service...

I think I'd want to have it for all connections. Sure, the majority will be graceful shutdowns (hopefully), but it's (I assume) easy enough to filter out a few uninteresting values and which values are uninteresting seems (as mentioned above) very subjective.

Another reason to include things that don't seem like errors is for making attack signatures. Something that's "normal" for a single connection may be problematic when it happens on many connections simultaneously. Or it may not be problematic, but still indicate that a change that signals a problem will soon occur.

Personally, I think I'd have just end_reason and drop error_type altogether (with the caveat that we might want a special way to represent unhandled exceptions).

If we go the other way, we have all sorts of strange cases to consider like whether cutting off a slow-reading client is an error. Arguably, that's the system functioning as expected and desired, but it's still interesting.

@amcasey
Copy link
Member

amcasey commented May 13, 2024

This seems like data that should be going into logs not metrics

@samsp-msft I'm not sure I'm following this line of reasoning. Given that logs are structured and queryable, why would anything not be logs-only? When would you use metrics instead?

@dotnet-policy-service dotnet-policy-service bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label May 21, 2024
@JamesNK JamesNK force-pushed the jamesnk/connection-protocol-code branch from eb1b413 to 6ed61b7 Compare June 10, 2024 03:44
@JamesNK JamesNK removed the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Jun 10, 2024
@dotnet-policy-service dotnet-policy-service bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Jun 27, 2024
@JamesNK JamesNK force-pushed the jamesnk/connection-protocol-code branch from 9b036f4 to 4c8f48d Compare June 28, 2024 14:04
@JamesNK JamesNK removed the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Jun 28, 2024
@JamesNK JamesNK force-pushed the jamesnk/connection-protocol-code branch 2 times, most recently from 4b4839f to c510461 Compare July 4, 2024 06:33
@dotnet-policy-service dotnet-policy-service bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Jul 11, 2024
src/Servers/Kestrel/Core/test/HttpResponseHeadersTests.cs Outdated Show resolved Hide resolved
src/Shared/Metrics/MetricsExtensions.cs Show resolved Hide resolved
src/Shared/Metrics/MetricsExtensions.cs Outdated Show resolved Hide resolved
@@ -2395,7 +2415,9 @@ public async Task ConnectionClosedAfter101Response()
[Fact]
public async Task ThrowingResultsIn500Response()
{
var testContext = new TestServiceContext(LoggerFactory);
var testMeterFactory = new TestMeterFactory();
using var connectionDuration = new MetricCollector<double>(testMeterFactory, "Microsoft.AspNetCore.Server.Kestrel", "kestrel.connection.duration");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider extracting a helper for new MetricCollector<double>(testMeterFactory, "Microsoft.AspNetCore.Server.Kestrel", "kestrel.connection.duration").

Copy link
Member

@amcasey amcasey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is big enough that I could keep giving feedback indefinitely, but I have no fundamental concerns and would be okay with merging it.

src/Servers/Kestrel/Core/src/Internal/Http3/Http3Stream.cs Outdated Show resolved Hide resolved
FlowControlWindowExceeded,
KeepAliveTimeout,
InsufficientTlsVersion,
InvalidHandshake,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems similar to UnexpectedFrame

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean?

InvalidHandshake is only used for HTTP/2 and occurs when the client doesn't start the connection with the HTTP/2 preface.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that comment was supposed to be about FrameAfterStreamClose. My bad.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I wonder if I was commenting on an older commit. I can't find a refresh button in the VS Code UI that seems to cover moving forward to the latest revision of the PR.

@@ -468,20 +473,24 @@ private void UpdateStreamTimeouts(long timestamp)
{
Log.RequestProcessingError(_context.ConnectionId, ex);
error = ex;
reason = ConnectionEndReason.IOError;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be fine to say it's some sort of transport level error.

}

public void OnInputOrOutputCompleted()
{
TryStopAcceptingStreams();

// Abort the connection using the error code the client used. For a graceful close, this should be H3_NO_ERROR.
Abort(new ConnectionAbortedException(CoreStrings.ConnectionAbortedByClient), (Http3ErrorCode)_errorCodeFeature.Error);
Abort(new ConnectionAbortedException(CoreStrings.ConnectionAbortedByClient), (Http3ErrorCode)_errorCodeFeature.Error, ConnectionEndReason.TransportCompleted);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like I mentioned in my other comment, I think indicating that this was client initiated would be good. If ClientFin is too TCP specific, we could go with ClientClosedTransport or something.

@JamesNK
Copy link
Member Author

JamesNK commented Jul 20, 2024

/benchmark plaintext aspnet-citrine-win kestrel

Copy link

pr-benchmarks bot commented Jul 20, 2024

Benchmark started for plaintext on aspnet-citrine-win with kestrel. Logs: link

Copy link

pr-benchmarks bot commented Jul 20, 2024

An error occurred, please check the logs

@JamesNK JamesNK removed the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Jul 20, 2024
@JamesNK
Copy link
Member Author

JamesNK commented Jul 20, 2024

/benchmark plaintext aspnet-citrine-win kestrel

Copy link

pr-benchmarks bot commented Jul 20, 2024

Benchmark started for plaintext on aspnet-citrine-win with kestrel. Logs: link

Copy link

pr-benchmarks bot commented Jul 20, 2024

plaintext - aspnet-citrine-win

application plaintext.base plaintext.pr
Max CPU Usage (%) 78 73 -6.41%
Max Cores usage (%) 2,170 2,038 -6.08%
Max Working Set (MB) 73 72 -1.37%
Max Private Memory (MB) 55 54 -1.82%
Build Time (ms) 3,715 3,842 +3.42%
Start Time (ms) 335 321 -4.18%
Published Size (KB) 107,273 107,273 0.00%
Symbols Size (KB) 54 54 0.00%
.NET Core SDK Version 9.0.100-preview.7.24369.5 9.0.100-preview.7.24369.5
Max CPU Usage (%) 79 71 -10.61%
Max Working Set (MB) 76 75 -0.49%
Max GC Heap Size (MB) 24 23 -1.67%
Size of committed memory by the GC (MB) 18 15 -19.73%
Max Number of Gen 0 GCs / sec 2.00 2.00 0.00%
Max Number of Gen 1 GCs / sec 1.00 1.00 0.00%
Max Number of Gen 2 GCs / sec 1.00 1.00 0.00%
Max Gen 0 GC Budget (MB) 19 20 +5.26%
Max Time in GC (%) 31.00 7.00 -77.42%
Max Gen 0 Size (B) 3,756,032 2,936,944 -21.81%
Max Gen 1 Size (B) 420,464 230,912 -45.08%
Max Gen 2 Size (B) 1,724,936 1,756,264 +1.82%
Max LOH Size (B) 121,000 121,000 0.00%
Max POH Size (B) 9,632,368 6,954,368 -27.80%
Max Allocation Rate (B/sec) 8,558,736 8,120,736 -5.12%
Max GC Heap Fragmentation (%) 742% 727% -2.05%
# of Assemblies Loaded 66 66 0.00%
Max Exceptions (#/s) 1,718 1,758 +2.33%
Max Lock Contention (#/s) 137 150 +9.49%
Max ThreadPool Threads Count 43 46 +6.98%
Max ThreadPool Queue Length 446 469 +5.16%
Max ThreadPool Items (#/s) 1,884,981 1,897,962 +0.69%
Max Active Timers 1 1 0.00%
IL Jitted (B) 223,730 223,573 -0.07%
Methods Jitted 2,965 2,975 +0.34%
load plaintext.base plaintext.pr
Max CPU Usage (%) 81 80 -1.23%
Max Cores usage (%) 2,277 2,254 -1.01%
Max Working Set (MB) 46 46 0.00%
Max Private Memory (MB) 370 370 0.00%
Build Time (ms) 3,465 3,559 +2.71%
Start Time (ms) 0 0
Published Size (KB) 72,283 72,283 0.00%
Symbols Size (KB) 0 0
.NET Core SDK Version 8.0.303 8.0.303
First Request (ms) 91 92 +1.10%
Requests/sec 9,955,598 9,999,865 +0.44%
Requests 150,316,336 150,997,558 +0.45%
Mean latency (ms) 34.77 49.03 +41.01%
Max latency (ms) 720.32 1,250.00 +73.53%
Bad responses 0 0
Socket errors 0 0
Read throughput (MB/s) 1,198.08 1,198.08 0.00%
Latency 50th (ms) 0.89 0.89 0.00%
Latency 75th (ms) 29.18 41.63 +42.67%
Latency 90th (ms) 119.44 171.25 +43.38%
Latency 99th (ms) 387.50 537.67 +38.75%

@JamesNK
Copy link
Member Author

JamesNK commented Jul 21, 2024

/benchmark

Copy link

pr-benchmarks bot commented Jul 21, 2024

Crank Pull Request Bot

/benchmark <benchmark[,...]> <profile[,...]> <component,[...]> <arguments>

Benchmarks:

  • plaintext: TechEmpower Plaintext Scenario - ASP.NET Platform implementation
  • plaintext_endpoint: TechEmpower Plaintext Scenario - ASP.NET Endpoint Routing
  • json: TechEmpower JSON Scenario - ASP.NET Platform implementation
  • fortunes: TechEmpower Fortunes Scenario - ASP.NET Platform implementation
  • yarp: YARP - http-http with 10 bytes
  • mvcjsoninput2k: Sends 2Kb Json Body to an MVC controller

Profiles:

  • aspnet-perf-lin: INTEL/Linux 12 Cores
  • aspnet-perf-win: INTEL/Windows 12 Cores
  • aspnet-citrine-lin: INTEL/Linux 28 Cores
  • aspnet-citrine-win: INTEL/Windows 28 Cores
  • aspnet-citrine-ampere: Ampere/Linux 80 Cores
  • aspnet-citrine-amd: AMD/Linux 48 Cores

Components:

  • kestrel
  • mvc
  • routing

Arguments: any additional arguments to pass through to crank, e.g. --variable name=value

@JamesNK
Copy link
Member Author

JamesNK commented Jul 22, 2024

/benchmark plaintext aspnet-citrine-win kestrel

Copy link

pr-benchmarks bot commented Jul 22, 2024

Benchmark started for plaintext on aspnet-citrine-win with kestrel. Logs: link

Copy link

pr-benchmarks bot commented Jul 22, 2024

plaintext - aspnet-citrine-win

application plaintext.base plaintext.pr
Max CPU Usage (%) 68 81 +19.12%
Max Cores usage (%) 1,903 2,266 +19.08%
Max Working Set (MB) 72 72 0.00%
Max Private Memory (MB) 54 54 0.00%
Build Time (ms) 3,279 3,293 +0.43%
Start Time (ms) 319 320 +0.31%
Published Size (KB) 107,273 107,273 0.00%
Symbols Size (KB) 54 54 0.00%
.NET Core SDK Version 9.0.100-preview.7.24371.4 9.0.100-preview.7.24371.4
Max CPU Usage (%) 68 96 +41.29%
Max Working Set (MB) 75 75 -0.02%
Max GC Heap Size (MB) 23 23 -0.51%
Size of committed memory by the GC (MB) 13 13 -5.83%
Max Number of Gen 0 GCs / sec 3.00 3.00 0.00%
Max Number of Gen 1 GCs / sec 2.00 2.00 0.00%
Max Number of Gen 2 GCs / sec 2.00 2.00 0.00%
Max Gen 0 GC Budget (MB) 17 18 +5.88%
Max Time in GC (%) 13.00 10.00 -23.08%
Max Gen 0 Size (B) 2,109,408 1,555,000 -26.28%
Max Gen 1 Size (B) 592,024 652,128 +10.15%
Max Gen 2 Size (B) 1,543,856 1,583,944 +2.60%
Max LOH Size (B) 121,000 121,000 0.00%
Max POH Size (B) 6,480,568 6,299,288 -2.80%
Max Allocation Rate (B/sec) 12,024,752 9,136,040 -24.02%
Max GC Heap Fragmentation (%) 641% 675% +5.40%
# of Assemblies Loaded 66 66 0.00%
Max Exceptions (#/s) 1,636 1,622 -0.86%
Max Lock Contention (#/s) 97 237 +144.33%
Max ThreadPool Threads Count 48 48 0.00%
Max ThreadPool Queue Length 390 303 -22.31%
Max ThreadPool Items (#/s) 1,868,889 1,917,623 +2.61%
Max Active Timers 1 1 0.00%
IL Jitted (B) 230,992 230,352 -0.28%
Methods Jitted 3,056 3,060 +0.13%
load plaintext.base plaintext.pr
Max CPU Usage (%) 82 81 -1.22%
Max Cores usage (%) 2,288 2,265 -1.01%
Max Working Set (MB) 46 46 0.00%
Max Private Memory (MB) 370 370 0.00%
Build Time (ms) 3,477 3,353 -3.57%
Start Time (ms) 0 0
Published Size (KB) 72,283 72,283 0.00%
Symbols Size (KB) 0 0
.NET Core SDK Version 8.0.303 8.0.303
First Request (ms) 92 92 0.00%
Requests/sec 9,798,015 9,894,495 +0.98%
Requests 147,952,480 149,398,880 +0.98%
Mean latency (ms) 22.60 65.35 +189.16%
Max latency (ms) 914.55 2,000.00 +118.69%
Bad responses 0 0
Socket errors 0 94 +∞%
Read throughput (MB/s) 1,177.60 1,187.84 +0.87%
Latency 50th (ms) 0.92 0.88 -4.35%
Latency 75th (ms) 12.34 33.75 +173.50%
Latency 90th (ms) 73.03 171.88 +135.36%
Latency 99th (ms) 288.06 1,320.00 +358.24%

@JamesNK JamesNK merged commit 4cf8947 into main Jul 22, 2024
26 checks passed
@JamesNK JamesNK deleted the jamesnk/connection-protocol-code branch July 22, 2024 02:34
@dotnet-policy-service dotnet-policy-service bot added this to the 9.0-preview7 milestone Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve Kestrel metrics
9 participants